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Introduction 

• What is this course about? 

• Schedule 

- What is Global Illumination? 

- Computing Global Illumination 

- Reusing Shading 

• Image-space 

• Mesh-based 

• Fast Gl 
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What is Global Illumination? 


Session I 


Assumptions: Geometric Optics 

• Light travels in straight lines 

• Rays do not interact with each other 

• Rays have color(wavelength), intensity 
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Reflections/Refractions 


• Interface between 2 materials 

reflection refraction 


• Specular reflections and refractions 
- One direction 
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Realistic Refiections/Refractions 


reflection refraction 
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Radiometric Terms 


• Power: energy per unit time 

• Irradiance: Incident power per unit surface area 

- From all directions 

- Watt/m2 

• Radiosity: Exitant power per unit surface area 

- Same units 
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Radiance 

• Radiance is radiant energy at x in direction 0: 
5D function 

- L{x — > 0) Power 

• per unit projected surface area 

• per unit solid angle 

d^P 

L{x -^e)= \ 

dA dA^ 

- units: Watt / m^.sr X 
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Why is radiance important? 

• Response of a sensor (camera, human eye) is 
proportional to radiance 

eye 

• Pixel values in image proportional to radiance 
received from that direction 
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Materiais - Three Forms 



Ideal diffuse 
(Lambertian) 


Ideal 

specular 


Directional 

diffuse 
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BRDF 


• Bidirectional Reflectance Distribution Function 



dLjx^e) 

’ dE{x<^^‘) Z(jc^^)cos(A^,,^)J(]G|j, 
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BRDF special case: ideal diffuse 

Pure Lambertian 


X(x,'P^©)= 

Tl 


Energy, „ 

<1 
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Rendering Equation (RE) 

• RE describes energy transport in scene 

• Input 

- Light sources 

- Surface geometry 

- Reflectance characteristics of surfaces 

• Output: value of radiance at all surface points in all 
directions 
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Rendering Equation 



L(x^0) = L^(x^0) + Z(x^0) 
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Rendering Equation 



hemisphere 
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Rendering Equation 



L(x->0) = 4(x^0) + 

j L(x^'F) /,(x,'Fo0)cos(N„^)^/co^ 

hemisphere 

Applicable for each wavelength 
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Rendering Equation 
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Computing Global Illumination 


Session II 


Classic Ray Tracing 

• Introduced in 1980 by Turner Whitted 

• Existing rendering: 

- Phong shading 

- Local illumination (specular, diffuse) 
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Insights 


• Trace rays from eye into scene 
- Backward ray tracing 


• Find visible objects 

• Shade visible points 

- Shadows 

- Reflections 

- Refractions 



Whitted 1980: First ray traced image 


• First global illumination algorithm! 
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Whitted RT Assumptions 

• Light Source: point light source 

- Hard shadows 

- Single shadow ray direction 

• Material: Blinn-Phong model 

- Diffuse with specular peak 

• Light Propagation 

- Occluding objects 

- Specular interreflections only 

• trace rays in mirror reflection direction only 
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History 


• Problems with classic ray tracing: 

- Not realistic: only perfect specular and perfect 
refractive/reflection between surfaces 

- View-dependent 

• Radiosity (1984) 

- Global Illumination in diffuse scenes 

- Discretize scene 

• Monte Carlo Ray Tracing (1986) 

- Global Illumination for any environment 
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Radiosity Advantages 


• Physically based approach for diffuse 
environments 

• Can model diffuse interactions, color bleeding, 
indirect lighting and penumbra (area light 
sources) 

• Boundary element (finite element) problem 

• Accounts for very high percentage of total 
energy transfer 
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Key Idea #1 : diffuse only 



• Radiance independent of direction 

• Surface looks the same from any viewpoint 

• No specular reflections 

GDC 2004 Copyright 2004 Kavita Bala and Bruce Walter, Cornell University 


Key Idea #2: “constanf’polygons 


• Radiosity solution is an approximation, due 
to discretization of scene into patches 
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Radiosity Equation 


• Radiosity equation for each polygon i 

N 

Radiosity^ - Radiosity + .^^Radiosity . 

7=1 

N 

Radiosity^ = Radiosity ^ ^ a.^^Radiosity . 

7=1 

N 

Radiosity - Radiosity ^ ^ a Radiosity j 

7=1 

• N equations; N unknown variables 
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Radiosity: Typical Image 



Rendering 
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Path Tracing 


• Monte Carlo Method 



- Randomly choose potential 


light paths 

- Average contribution over 
many such paths 

Problems 

- Noise 

- Slow to converge 
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Approaches 

• Bidirectional 

• Metropolis 

• Photon Mapping 

• Instant Radiosity 
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Fast Global Illumination 


Session III 


Hardware’s Strengths 

• What is the hardware good at? 

- Fast visibility determination (z-buffer) 

- Fast texture map lookup 

- Fast shading 

• Can even be per-pixel with latest boards 


GDC 2004 Copyright 2004 Kavita Bala and Bruce Walter, Cornell University 


17 









Hardware’s Strengths 

• Fast visibility determination (z-buffer) 

- Fast in an amortized sense 

• One rendering determines the visible surface seen at 
all pixels simultaneously 

- Great for some visibility queries types 

• Primary (eye) rays 

• Shadow rays (point sources) 

- Not SO good for other types 

• Reflection & refraction from curved surfaces 

• Indirect illumination 

• Adaptive sampling 
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Hardware’s Strengths 


• Fast texture map lookup 

- Cheap anti-aliasing & anisotropic filtering 

- Most flexible part of graphics hardware 

- Surface texturing 

- Bump mapping 

- Reflection mapping 

- Shadow mapping 

- Even arbitrary BRDF approximations 

- Very powerful 

• But not as compute intensive 
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Hardware’s Strengths 

• Fast shading 

- Latest boards can do per-pixel shading 

- Programmable, but limited operations 

- Local shading only 

• Aii inputs must be provided ahead of time 

• Non-iocai shading can oniy be approximated 

- Shadows, reflections, indirect, etc 
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Hardware’s Strengths 


• Conclusion 

- Hardware still has the edge due to its dedicated 
pipeline 

- Advantage may actually be decreasing 

- Software attractive for its flexiblility 

• If it can be made “fast enough” for interactive use 

• And handle scene and/or effects the hardware cannot handle 
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Interactive Software Rendering 

• Interactive 

- User-driven, not pre-scripted animation 

- At least a few frames per second (fps) 

• Software 

- Major shading done in software 

• Can use hardware to help 

• Rendering 

- Online, not pre-computed or captured 

• Eg, lightfields are pre-computed 
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Ray Tracing (or Ray Casting) 


• Very common visibility tool for software 

- Flexible 

- Efficient for large models 

• Using an acceleration structure (grids, bsp, etc) 

- Usually the largest computational bottleneck 

- Easily parallelizable 

• Each pixel can be computed in parallel 
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Interactive RT 

• SGI Origin 2000 

- 64 processors 

- Shared memory 

• Whitted-style ray tracing 

- Shadow, reflection, and 
refraction rays 

• Non-polygonal primitives 

- Spheres and splines 
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[Parker et al.] 



15 fps 

Bruce Walter, Cornell University 
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Coherent RT [Wald et. al.] 

• Created a highly optimized ray tracing engine for 
Intel-based PCs 

- Carefully profiled their ray tracer 

• Discovered it was often memory bound 

- Hand-crafted and tuned their code 

• Created both C and assembly versions 

• Used compact, cache-friendly data structures 

• Optimized for SIMD (SSE) 

• Reordered computations for better coherence 
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Test Scenes I 


• Test scenes: 800 triangles to >8 million 



GDC 2004 Copyright 2004 Kavita Bala and Bruce Walter, Cornell University 


Test Scenes II 

• Test scenes: 800 triangles to >8 million 

Berkeley Soda Hall: 
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Performance Results III 


• Comparison to Rasterization-Hardware 

- Ray tracing scales well for large environments 
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Fast RT [Benthin et. al.] 


- SIMD and careful hand coding of inner loops 
-0.4-2 Million rays per second 

• Athlon MP 1800+ 

- Instant Radiosity used for global illumination 

- Up to 48 processors for interactive performance 
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Reusing Shading for Giobai 
iilumination 


Session IV 







Reusing Shading 


• Full simulation can take 
hours 


• Interactive rendering by 
exploiting coherence 
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Program of Computer Graphics 
Cornell University 


Dispiay Process 

• Automatically exploit spatial and temporal 
coherence 

• Layered on top of an existing (slow) global 
illumination renderer 

• Provide interactive performance 
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Interactive Global Illumination 


• Standard visual 
feedback loop 

- Entirely 
synchronous 

- Framerate is 
limited by the 
Tenderer 


image 
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Interactive Global Illumination 


• Modified visual 


Image 



interface 
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Interactive Global Illumination 


• Interactive requirements 

- Image quality 

- Responsiveness 

• Don’t make the user wait 

• Provide rapid user feedback 

- Consistency 

• Don’t surprise or distract the user 

• Avoid sudden changes if possibie 

- Eg, in quality, frame rate, popping, etc, 
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Issues 

• How to cache results for reuse? 

- Interpolate images from sparse data 

- Handle camera motion between frames 

• Handie occiusion changes 

• Which samples should be rendered? 

- Prioritize for maximum benefit 

• What if scene or shading changes? 

- Detect and discard data that is no longer valid 
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Approaches 


• Image based 

- Post-rendering Warp (Mark97) 

- Corrective texturing (StammingerOO) 

• Point based 

- Render Cache (Walter99,02) 

- Edge-and-Point Rendering (BalaOS) 

• Mesh based 

- Tapestry (SimmonsOO) 

- Shading Cache (Tole02) 

• 4D approaches 

- Holodeck (Ward98,99) 

- Radiance Interpolants (Bala99) 

GDC 2004 Copyright 2004 Kavita Bala and Bruce Walter, Cornell University 


Frameless Rendering 

• Update pixels as they are computed 
- Don’t wait for full frame to finish 



timje ( ms) 0 | | 2^0 | | 400 


F^Tiie 1. 'TTie bottom ion? diows? ftamjesofa. 15 Hi ftamelessTeiudeTiTigsieqTijenceTiritli 33% of the pixels 
Tipdatied in eijcli fume. The middle totv shows 3 fumes of a dotibOe-btiffeiL'na seqnenjoe tipdaled at 5 Hi. 
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Post-Rendering 3D Warp 


• Render subset of frames 

- E.g, every 6th frame is rendered 

• Use standard image warping techniques to 
compute the other frames 
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Post-Rendering 3D Warp 


• Problem: holes and missing data 



Reference frame Warped frame 


The camera is moving to the left in this example. 
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Post-Rendering 3D Warp 


• Warp from both past and future reference frames 
- Heuristics for combining pixel results 



Prior reference Warped frame Next reference 
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Post-Rendering 3D Warp 


Must predict the locations of future frames 
- Longer predictions become rapidly less accurate 



GDC 2004 


— Camera path 
O Warped frames 
• Predicted frames 
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Corrective Texturing 


• Start with a standard hardware rendering of scene 

- Graphics hardware very good at interactive display 

- Start with a radiosity solution 

• Compare to underlying renderer 

- Apply corrections where they differ 

- Corrections applied as projective textures 
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Corrective Texturing 


Corrective textures are 
dynamically assigned to 
objects I 
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Corrective Texturing 



Radiosity Corrected Corrective 

solution image texture 
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Corrective Texturing 

• Sparse rendered samples compared to hardware 
displayed results 

- Differences splatted into textures 

- More samples generated near points which had large 
differences 

- Samples which are likely to have changed are 
deprecated so that can be overwritten by future results 
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Approaches 


• Image based 

- Post-rendering Warp (Mark97) 

- Corrective texturing (StammingerOO) 

• Point based 

- Render Cache (Walter99,02) 

- Edge-and-Point Rendering (BalaOS) 

• Mesh based 

- Tapestry (SimmonsOO) 

- Shading Cache (Tole02) 

• 4D approaches 

- Holodeck (Ward98,99) 

- Radiance Interpolants (Bala99) 
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Render Cache 

• Results stored as cloud of unordered points with 

- 3D position (located on surfaces) 

- Color 
-Age 

- Object identifier 
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Render Cache 


• Reproject points into current frame 

- Occlusion errors 

- Holes in data 



Initial view After reprojection 
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Render Cache 


Use occlusion culling heuristic 
Interpolation to fill holes 
- Fixed size kernels, 3x3 and 7x7 


Reprojection Occlusion cull Interpolation 
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Render Cache 


• Priority image for sampling 

- High priority for sparse regions 

- High priority for old points 

• Convert priority image to sparse set of locations to 
be rendered 

- Uses error-diffusion dither 

• Also uses predictive sampling 

- Try to sample new regions just before they become 
visible 
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Render Cache 


Displayed image Priority image Requested pixels 
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Render Cache 


• Recomputes old samples to detect changes 

- Nearby points are aged to raise priority and cause 
point invalidation 

• Object motion 

- Associated points can be transformed along with the 
object 
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Edge-and-Point Rendering 


S Edges: important discontinuities 

- Silhouettes and shadows 

Points: sparse shading samples 


edge -and -point 
reconstruction 
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Edge-and-Point Image 


Alternative display representation 

Edge-constrained interpolation preserves sharp 
features 

Fast anti-aliasing i 




-f — 


o_ 








n 











b 
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Pixel types 

• Pixels can have arbitrary edge complexity 

• Classify pixels into 3 groups 

- Empty: no edges 

- Simple: can be approximated by 1 edge 

- Complex: everything else 



Empty Simple Complex 


- Typical pixel classification statistics 

• empty (85-95%), simple (4-10%), complex (1-4%) 
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Edge-and-Point Image (EPI) 


• Goal: compact and fast 

- Store at most one edge 

and one point per pixel 

- Limited sub-pixel 

precision 


• Combine edges and points in image space 
- View-driven, lazy evaluation 
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Fast Anti-aliasing 



Magnified view of a ray traced 
image with 1 sampie per pixei 


Our resuit using 
<1 sampie per pixei 
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Putting it Together 



Resuits: Quaiity 


• Global Illumination 

- 3 lights 

- 1 50k polygons 

• Sparseness Ratio 

- 100: 1 

• Performance 



- 8-14 fps 


Without Edges With Edges 
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Approaches 


• Image based 

- Post-rendering Warp (Mark97) 

- Corrective texturing (StammingerOO) 

• Point based 

- Render Cache (Walter99,02) 

- Edge-and-Point Rendering (BalaOS) 

• Mesh based 

- Tapestry (SimmonsOO) 

- Shading Cache (Tole02) 

• 4D approaches 

- Holodeck (Ward98,99) 

- Radiance Interpolants (Bala99) 

GDC 2004 Copyright 2004 Kavita Bala and Bruce Walter, Cornell University 


Holodeck 

• Uses Radiance as its renderer 

• Rendered samples stored in a 4D data structure 

- Similar to Light Field or Lumigraph 

- Can be very large 

• Paged to disk if necessary 

- Lazily evaluated 

• Samples generated near current viewpoint 

- Position and other parameters are specified by the 
user 
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Holodeck 


• Uses Gouraud-shaded triangle mesh 

- Get samples near current viewpoint 

- Samples become vertices in a mesh 

- Delaunay triangulation of samples in direction space 
about a center of projection 

- Hardware provides fast display including interpolation 
between samples 
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Holodeck 

• Mesh construction 

- Choose center point 

- Construct Delaunay triangulation 

• Based on sample point’s projection onto a sphere about the 
center point 

- Display mesh using hardware 

- Update incrementally with new samples 

• If user moves too far, then must choose new 
center and rebuild mesh 
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Holodeck 


• Depth heuristic to reduce occlusion errors 

• Special techniques for designated moving objects 
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Tapestry 

• Based on Holodeck system with several 
enhancements: 

- Prioritized sampling 

- Incremental “recentering” of spherical Delaunay mesh 
as viewpoint moves 

- Fixed cache size 

• Max vertices = pixels 

- Sample invalidation 

• Occlusion and color change heuristics 
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Tapestry 


• Each triangle assigned a priority 

- Color & depth differences and age 

- Rasterize priority using hardware 

- Quasi-random sampling with rejection 



Image Priority Samples 
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Shading Cache 

• Display mesh is refinement of original scene mesh 

- No occlusion errors 

- Hardware handles textures 

- Display mesh >= original mesh 

- Easier to handle moving objects 

• Decouples frame update from mesh update 
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Shading Cache 


• Adds flood-fill heuristic for sampling 

- Discontinuities require locally dense subdivision 

• Mesh de-refinement 

- If not recently visible 

- If denser than pixel spacing 

- If color changes are detected 
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Shading Cache 
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Radiance Interpolants Bala[96,97,99] 


• Radiance interpolant: 

- Set of sparse samples (in 4D line space) of radiance 
function 

- Bounded-error, accurate reconstruction 
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Results: Museum Scene 






gray: interpolation success 
yellow: silhouettes; green: shadows 
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Downloadable Versions 


• Render Cache 

- http://www.graphics.cornell.edu/research/interactive/re 
ndercache/ 

• Holodeck 

- http ://radsite. Ibl . gov/radiance 
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Conclusion 

• Software Interactive Rendering is possible now 
with current machines 

- Good scaling with scene complexity 

- Much greater shading flexibility 

• Many interesting challenges still remain 

- Higher resolutions 

- Fully dynamic environments 

- Anti-aliasing 
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